feat: push logic for checkpoints v2 refs by pfleidi · Pull Request #821 · entireio/cli

pfleidi · 2026-03-31T22:23:45Z

Summary

Implements push logic for checkpoints v2 refs under refs/entire/. This is task A7 from the checkpoints v2 design spec.

When both checkpoints_v2 and push_v2_refs are enabled in strategy options, the pre-push hook pushes v2 refs alongside the existing v1 branch using explicit refspecs:

refs/entire/checkpoints/v2/main:refs/entire/checkpoints/v2/main
refs/entire/checkpoints/v2/full/current:refs/entire/checkpoints/v2/full/current
refs/entire/checkpoints/v2/full/0000000000001:refs/entire/checkpoints/v2/full/0000000000001

How it works

Push gating: Both settings must be true:

{
  "strategy_options": {
    "checkpoints_v2": true,
    "push_v2_refs": true
  }
}

What gets pushed: /main, /full/current, and the latest archived generation (older archives are immutable and were pushed when created).

Merge recovery for /main and /full/current: When a push fails (non-fast-forward), the CLI fetches the remote ref, flattens both trees, combines entries, and creates a merge commit — same strategy as v1. This works conflict-free because both refs contain only sharded checkpoint directories (<id[:2]>/<id[2:]>/) with no root-level files.

Rotation conflict recovery: When /full/current push fails and the remote has an archived generation we don't have locally, another machine rotated. Instead of merging two divergent /full/current refs (which would duplicate data across generations), the CLI merges local data into the latest archived generation:

Starting state (synced): /full/current has 95 checkpoints

User A: adds 7 → 102 → rotation triggers
  /full/0000000000001: 102 checkpoints (95 shared + 7 from A)
  /full/current: fresh orphan
  Pushed to remote.

User B: adds 3 → 98 locally, pushes /full/current → non-fast-forward
  Recovery:
    1. ls-remote → discovers /full/0000000000001 (not local)
    2. Fetch /full/0000000000001
    3. Tree-merge local /full/current into /full/0000000000001
       → 95 shared deduplicated by git content-addressing, result: 105
    4. Update generation.json timestamps, push /full/0000000000001
    5. Adopt remote /full/current as local

Final state:
  /full/0000000000001: 105 checkpoints (no duplication)
  /full/current: fresh orphan

This preserves the key GC property: each checkpoint's raw transcript exists in exactly one generation, so deleting a generation ref makes its unique blobs unreachable.

generation.json changes

generation.json is no longer written to /full/current on every checkpoint write. It's written only at archive time (during rotation), containing just timestamps:

{
  "oldest_checkpoint_at": "2026-01-01T00:00:00Z",
  "newest_checkpoint_at": "2026-02-15T00:00:00Z"
}

The checkpoints list field has been removed. Checkpoint count for rotation is determined by walking shard directories in the tree. This keeps /full/current free of root-level files, making all tree merges conflict-free.

Fetch-on-demand

Checkpoint remote: When push_v2_refs is enabled and a checkpoint remote is configured, the v2 /main ref is fetched from the checkpoint remote URL on first push (same one-time pattern as v1).
entire resume: When a transcript isn't found locally, readTranscriptFromFullRefs discovers and fetches remote /full/* refs before giving up. Respects checkpoint_remote configuration.
All V2GitStore instances receive the resolved fetch remote at construction time — no hardcoded "origin".

Other improvements

Extracted WalkCheckpointShards helper for reusable two-level shard iteration (used by CountCheckpointsInTree and ListCommitted)
Exported GenerationRefPattern, GetRefState, ListArchivedGenerations, AddGenerationJSONToTree for cross-package use

Test plan

Unit tests for all push/merge/rotation functions (push_v2_test.go)
Integration tests for full push cycle and disabled-push gating (v2_push_test.go)
Updated generation tests for archive-time-only generation.json
mise run fmt && mise run lint && mise run test:ci passes

Note

Medium Risk
Adds new git push/fetch/merge paths for custom v2 refs (including rotation-conflict handling) and changes generation rotation metadata, which could impact checkpoint durability and resume/restore behavior if edge cases are missed.

Overview
Enables optional pushing of v2 checkpoint refs (under refs/entire/checkpoints/v2/*) from the pre-push hook when both checkpoints_v2 and push_v2_refs are set, including fetch+tree-merge recovery on non-fast-forward and special handling when /full/current was rotated remotely.

Changes v2 generation management so /full/current no longer writes generation.json on each write; rotation now counts checkpoints by walking shard dirs, writes generation.json only when archiving, and resets /full/current to an empty orphan commit.

Adds v2 read/fetch plumbing: V2GitStore can read committed metadata and raw transcripts across /full/current + archived generations (with on-demand remote fetch), resume/RestoreLogsOnly prefer v2 when enabled with v1 fallback, and checkpoint-remote setup can fetch the v2 /main ref when v2 pushing is enabled.

^{Written by Cursor Bugbot for commit edca0a9. Configure here.}

…io/cli into feat/checkpoints-v2-push-logic

Entire-Checkpoint: ae2c70521c60

…chive time generation.json is no longer written to /full/current on every write. This keeps /full/current free of root-level files, ensuring conflict-free tree merges during push recovery. Rotation threshold is now checked by counting shard directories in the tree. - Remove Checkpoints field from GenerationMetadata (timestamps only) - Replace updateGenerationForWrite with CountCheckpointsInTree - Write generation.json only during rotateGeneration (archive time) - Fresh /full/current orphan has empty tree (no generation.json) - Export GetRefState, ListArchivedGenerations, AddGenerationJSONToTree Entire-Checkpoint: 76722c974e37

Reusable helper that walks the <id[:2]>/<id[2:]>/ shard structure, replacing hand-rolled two-level loops in CountCheckpointsInTree and ListCommitted. Entire-Checkpoint: efb9c422b636

Ref-aware push functions that use explicit refspecs for refs under refs/entire/. No remote-tracking ref optimization — always attempts the push and lets git handle the no-op case. fetchAndMergeRef is stubbed for now (implemented next). Entire-Checkpoint: 191b34cb4fe7

Tree-flattening merge for custom refs under refs/entire/. Uses temp refs for fetching and the same sharded-path merge strategy as v1. Entire-Checkpoint: bd33aa361f7e

pushV2Refs pushes /main, /full/current, and the latest archived generation. Gated by IsPushV2RefsEnabled (requires both checkpoints_v2 and push_v2_refs settings). Older archived generations are immutable and were pushed when created. Entire-Checkpoint: fc4cb56be142

When remote /full/current was rotated by another machine, merges local checkpoints into the latest archived generation instead of creating duplicates. Git content-addressing deduplicates shared checkpoint data. Uses local commit timestamps for generation.json (not time.Now()) so cleanup scheduling reflects actual checkpoint creation time. Entire-Checkpoint: 6dc060481ed1

Extends resolvePushSettings to also fetch the v2 /main ref from the checkpoint remote URL when push_v2_refs is enabled. Same one-time fetch pattern as the v1 metadata branch. Entire-Checkpoint: 7c81d2786447

When a transcript isn't found locally, readTranscriptFromFullRefs now discovers and fetches remote /full/* refs from origin before giving up. Only newly fetched refs are searched on the second pass. Entire-Checkpoint: 6b193c4a3527

Verifies that PrePush pushes v2 refs to remote when push_v2_refs is enabled, and skips them when disabled. Both tests use the full CLI hook path (SimulateUserPromptSubmit → Stop → Commit → RunPrePush). Entire-Checkpoint: 199d3876a229

- Wrap errors from external packages (wrapcheck) - Add nolint explanations (nolintlint) - Suppress unchecked pushRefIfNeeded returns (errcheck) - Simplify computeGenerationTimestamps: remove unused error return (unparam, nilerr) - Fix wasted assignment in commit walk loop (wastedassign) - Fix gofmt alignment Entire-Checkpoint: 14886a520190

Export GenerationRefPattern from checkpoint package and reuse in strategy/push_v2.go instead of defining an identical regex in both. Entire-Checkpoint: 3c0dac7843a9

…on recovery Entire-Checkpoint: dbea736f4572

Entire-Checkpoint: ef2eca715f94

NewV2GitStore now requires a fetchRemote parameter used for fetch-on-demand operations. All callers resolve the checkpoint remote consistently via resolveV2FetchRemote (strategy package) or ResolveCheckpointURL (exported for resume.go). getV2CheckpointStore now accepts context so settings are loaded from the correct working directory. Entire-Checkpoint: e976eeced1a6

Entire-Checkpoint: 551f42342598

…inRefIfMissing Entire-Checkpoint: da8bea2a421e

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

✅ Fixed: Inconsistent JSON serialization for generation.json during rotation recovery
- I replaced json.Marshal with jsonutil.MarshalIndentWithNewline in updateGenerationTimestamps so rotation recovery writes generation.json with the same formatting as other code paths.

Or push these changes by commenting:

@cursor push 74c23307e6

Preview (74c23307e6)

diff --git a/cmd/entire/cli/strategy/push_v2.go b/cmd/entire/cli/strategy/push_v2.go
--- a/cmd/entire/cli/strategy/push_v2.go
+++ b/cmd/entire/cli/strategy/push_v2.go
@@ -14,6 +14,7 @@
 	"time"
 
 	"github.com/entireio/cli/cmd/entire/cli/checkpoint"
+	"github.com/entireio/cli/cmd/entire/cli/jsonutil"
 	"github.com/entireio/cli/cmd/entire/cli/logging"
 	"github.com/entireio/cli/cmd/entire/cli/paths"
 
@@ -367,7 +368,7 @@
 		gen.NewestCheckpointAt = newestFromLocal
 	}
 
-	updatedData, err := json.Marshal(gen)
+	updatedData, err := jsonutil.MarshalIndentWithNewline(gen, "", "  ")
 	if err != nil {
 		return object.TreeEntry{}, fmt.Errorf("failed to marshal generation.json: %w", err)
 	}

_{This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.}

Comment @cursor review or bugbot run to trigger another review on this PR

cmd/entire/cli/strategy/push_v2.go

cmd/entire/cli/checkpoint/committed.go

Copilot

Pull request overview

Adds v2 checkpoint ref syncing under refs/entire/ (push + conflict recovery) and wires v2 reads into resume / log restore paths so v2 metadata/transcripts can be used when enabled.

Changes:

Implement v2 ref push with non-fast-forward recovery (tree merge + rotation-aware recovery for /full/current).
Add v2 read paths for resume/restore (v2-first with v1 fallback), including fetch-on-demand for /full/*.
Update v2 generation handling to keep /full/current root clean (archive-time-only generation.json; shard-walk counting).

Reviewed changes

Copilot reviewed 29 out of 32 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
cmd/entire/cli/strategy/push_v2.go	New v2 push + fetch/merge recovery logic (including rotation conflict handling).
cmd/entire/cli/strategy/push_v2_test.go	Unit tests covering push/merge/rotation recovery behavior.
cmd/entire/cli/strategy/manual_commit_push.go	Hook now optionally pushes v2 refs when gated by settings.
cmd/entire/cli/strategy/checkpoint_remote.go	Adds v2 “fetch /main if missing” and exposes checkpoint URL resolution for reuse.
cmd/entire/cli/settings/settings.go / settings_test.go	Adds `push_v2_refs` gating (requires `checkpoints_v2`).
cmd/entire/cli/resume.go	Resume prefers v2 metadata/log lookup when enabled, with v1 fallback.
cmd/entire/cli/git_operations.go	Adds v2 `/main` fetch helpers (treeless + full).
cmd/entire/cli/checkpoint/v2_store.go (+ tests)	V2GitStore now carries a configured fetch remote and exports more helpers.
cmd/entire/cli/checkpoint/v2_read.go (+ tests)	Implements v2 committed/session reads and transcript discovery across `/full/*` refs.
cmd/entire/cli/checkpoint/v2_generation.go (+ tests)	generation.json semantics updated; shard-walk counting + exported generation helpers.
cmd/entire/cli/checkpoint/parse_tree.go / committed.go	Introduces reusable shard walker and uses it for committed listing.
cmd/entire/cli/integration_test/v2_resume_test.go / v2_push_test.go	Integration coverage for v2 resume and v2 ref push gating/cycle.

Comments suppressed due to low confidence (1)

cmd/entire/cli/resume.go:475

getV2MetadataTree injects FetchV2MainTreeOnly/FetchV2MainRef, which currently fetch from hardcoded "origin". When a checkpoint_remote is configured, v2 refs are pushed to that URL instead of origin, so resume’s v2 metadata lookup can fail even though the data exists remotely. Consider wiring the resolved checkpoint remote (or push remote) into the v2 metadata fetch functions so they fetch from the same remote used for pushing checkpoints.

func getV2MetadataTree(ctx context.Context) (*object.Tree, *git.Repository, error) {
	tree, repo, err := checkpoint.GetV2MetadataTree(ctx, FetchV2MainTreeOnly, FetchV2MainRef, openRepository)
	if err != nil {
		return nil, nil, fmt.Errorf("v2 metadata tree: %w", err)
	}
	return tree, repo, nil

cmd/entire/cli/strategy/push_v2.go

Copilot · 2026-03-31T22:30:01Z

cmd/entire/cli/strategy/push_v2.go

+	tmpRefName := plumbing.ReferenceName("refs/entire-fetch-tmp/" + tmpRefSuffix)
+	refSpec := fmt.Sprintf("+%s:%s", refName, tmpRefName)
+
+	fetchCmd := exec.CommandContext(ctx, "git", "fetch", target, refSpec)


This git fetch runs inside the pre-push hook but doesn't disconnect stdin (and doesn't set GIT_TERMINAL_PROMPT=0). Other hook-side git commands in this codebase set fetchCmd.Stdin = nil to avoid credential prompts/hangs in hook context. Please align this fetch with that pattern.

Suggested change

fetchCmd := exec.CommandContext(ctx, "git", "fetch", target, refSpec)

fetchCmd := exec.CommandContext(ctx, "git", "fetch", target, refSpec)

fetchCmd.Stdin = nil

cmd/entire/cli/strategy/push_v2.go

cmd/entire/cli/checkpoint/v2_store.go

…s-v2-push-logic

Entire-Checkpoint: 3f733012225d

… git commands Entire-Checkpoint: 5a39329befe8

… paths Entire-Checkpoint: ca27580dd7cc

…on.json formatting Entire-Checkpoint: b9153880ad4c

Entire-Checkpoint: fbfec65420f9

Entire-Checkpoint: b24d67ac16b5

…ush-logic # Conflicts: # cmd/entire/cli/git_operations.go # cmd/entire/cli/resume.go

…ush-logic

Aligns with v1 push path which uses CheckpointGitCommand to inject ENTIRE_CHECKPOINT_TOKEN for authenticated git operations. Previously v2 used raw exec.CommandContext which skipped token injection. Entire-Checkpoint: ef14af5c9122

…e' into feat/checkpoints-v2-push-logic # Conflicts: # cmd/entire/cli/checkpoint/v2_read.go # cmd/entire/cli/git_operations.go

…FromURL Removes duplicated fetch logic — now checks if ref exists locally and delegates to FetchV2MainFromURL (from resume branch) which uses CheckpointGitCommand for token injection. Entire-Checkpoint: 6e0818a62512

Entire-Checkpoint: 97687717cb39

Entire-Checkpoint: 95e4f0b990a0

cmd/entire/cli/checkpoint/v2_generation.go

computermode · 2026-04-03T21:45:55Z

cmd/entire/cli/resume.go

+			fetchRemote := strategy.ResolveCheckpointURL(ctx, "origin")
+			if fetchRemote == "" {
+				fetchRemote = "origin"
+			}


Maybe this is a nitpick, but calling out that we seem to be duplicating logic here as what's in resolveV2FetchRemote in checkpoint_remote.. maybe we want to export that and reuse it here so we don't drift / have to maintain it in two places

computermode · 2026-04-03T21:46:55Z

cmd/entire/cli/checkpoint/v2_store.go

+func NewV2GitStore(repo *git.Repository, fetchRemote string) *V2GitStore {
+	if fetchRemote == "" {
+		fetchRemote = "origin"
+	}


Same comment here, re: duplicating some logic

(I'm finding these because I wanted to make sure we were still supporting external repos for storing the checkpoints.. looks like we are if I am understanding the logic correctly)

pfleidi added 19 commits March 27, 2026 18:03

Merge branch 'feat/checkpoints-v2-entire-resume' of github.com:entire…

7201db1

…io/cli into feat/checkpoints-v2-push-logic

Merge branch 'feat/checkpoints-v2-entire-resume' of github.com:entire…

60fbfdc

…io/cli into feat/checkpoints-v2-push-logic

feat: Introduce IsPushV2RefsEnabled()

e71bfdb

Entire-Checkpoint: ae2c70521c60

refactor: extract WalkCheckpointShards helper for shard iteration

4ebd0b4

Reusable helper that walks the <id[:2]>/<id[2:]>/ shard structure, replacing hand-rolled two-level loops in CountCheckpointsInTree and ListCommitted. Entire-Checkpoint: efb9c422b636

feat: implement fetchAndMergeRef for v2 custom ref merge recovery

92c4f7b

Tree-flattening merge for custom refs under refs/entire/. Uses temp refs for fetching and the same sharded-path merge strategy as v1. Entire-Checkpoint: bd33aa361f7e

feat: fetch v2 /main ref from checkpoint remote when missing locally

186271d

Extends resolvePushSettings to also fetch the v2 /main ref from the checkpoint remote URL when push_v2_refs is enabled. Same one-time fetch pattern as the v1 metadata branch. Entire-Checkpoint: 7c81d2786447

test: add integration tests for v2 push cycle

5f79093

Verifies that PrePush pushes v2 refs to remote when push_v2_refs is enabled, and skips them when disabled. Both tests use the full CLI hook path (SimulateUserPromptSubmit → Stop → Commit → RunPrePush). Entire-Checkpoint: 199d3876a229

refactor: deduplicate generationRefPattern across packages

05ee59e

Export GenerationRefPattern from checkpoint package and reuse in strategy/push_v2.go instead of defining an identical regex in both. Entire-Checkpoint: 3c0dac7843a9

fix: log warning when generation timestamp update fails during rotati…

13fd367

…on recovery Entire-Checkpoint: dbea736f4572

fix: document why fetchRemoteFullRefs uses origin directly

ca7d12b

Entire-Checkpoint: ef2eca715f94

fix: use plumbing.ReferenceName equality instead of string cast

dd13101

Entire-Checkpoint: 551f42342598

fix: add nolint explanations for intentional nil returns in fetchV2Ma…

edca0a9

…inRefIfMissing Entire-Checkpoint: da8bea2a421e

pfleidi requested a review from a team as a code owner March 31, 2026 22:23

Copilot AI review requested due to automatic review settings March 31, 2026 22:23

Copilot started reviewing on behalf of pfleidi March 31, 2026 22:24 View session

cursor bot reviewed Mar 31, 2026

View reviewed changes

cmd/entire/cli/strategy/push_v2.go Outdated Show resolved Hide resolved

pfleidi commented Mar 31, 2026

View reviewed changes

cmd/entire/cli/checkpoint/committed.go Outdated Show resolved Hide resolved

pfleidi changed the base branch from main to feat/checkpoints-v2-entire-resume March 31, 2026 22:29

Copilot AI reviewed Mar 31, 2026

View reviewed changes

pfleidi added 4 commits March 31, 2026 15:34

Merge branch 'feat/checkpoints-v2-entire-resume' into feat/checkpoint…

ac79fb7

…s-v2-push-logic

fix: resolve lint warnings (errcheck, nilerr, testifylint)

6829af1

Entire-Checkpoint: 3f733012225d

review: disconnect stdin and disable terminal prompts on hook-context…

09ad6c7

… git commands Entire-Checkpoint: 5a39329befe8

review: defer temp ref cleanup in fetchAndMergeRef to cover all error…

55538f5

… paths Entire-Checkpoint: ca27580dd7cc

pfleidi and others added 9 commits March 31, 2026 16:23

review: use jsonutil.MarshalIndentWithNewline for consistent generati…

cf0e643

…on.json formatting Entire-Checkpoint: b9153880ad4c

review: default fetchRemote to origin when empty in NewV2GitStore

664b843

Entire-Checkpoint: fbfec65420f9

fix: remove unused nolint:nilerr directives in fetchV2MainRefIfMissing

61d1648

Entire-Checkpoint: b24d67ac16b5

Merge remote-tracking branch 'origin/main' into feat/checkpoints-v2-p…

aeadae0

…ush-logic # Conflicts: # cmd/entire/cli/git_operations.go # cmd/entire/cli/resume.go

Merge remote-tracking branch 'origin/main' into feat/checkpoints-v2-p…

4945b1c

…ush-logic

Merge remote-tracking branch 'origin/feat/checkpoints-v2-entire-resum…

fc1aa48

…e' into feat/checkpoints-v2-push-logic # Conflicts: # cmd/entire/cli/checkpoint/v2_read.go # cmd/entire/cli/git_operations.go

review: add debug logging for checkpoint_remote URL resolution failures

2bea35f

Entire-Checkpoint: 97687717cb39

Base automatically changed from feat/checkpoints-v2-entire-resume to main April 2, 2026 22:49

pfleidi added 2 commits April 3, 2026 13:46

Merge branch 'main' into feat/checkpoints-v2-push-logic

962675a

fix: add missing fetchRemote arg to NewV2GitStore calls from merged main

d3531ce

Entire-Checkpoint: 95e4f0b990a0

computermode reviewed Apr 3, 2026

View reviewed changes

cmd/entire/cli/checkpoint/v2_generation.go Show resolved Hide resolved

computermode reviewed Apr 3, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: push logic for checkpoints v2 refs#821

feat: push logic for checkpoints v2 refs#821
pfleidi wants to merge 34 commits intomainfrom
feat/checkpoints-v2-push-logic

pfleidi commented Mar 31, 2026 •

edited by cursor bot

Loading

Uh oh!

cursor bot left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI Mar 31, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

computermode Apr 3, 2026

Uh oh!

computermode Apr 3, 2026

Uh oh!

computermode Apr 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

	fetchCmd := exec.CommandContext(ctx, "git", "fetch", target, refSpec)
	fetchCmd := exec.CommandContext(ctx, "git", "fetch", target, refSpec)
	fetchCmd.Stdin = nil

Conversation

pfleidi commented Mar 31, 2026 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How it works

generation.json changes

Fetch-on-demand

Other improvements

Test plan

Uh oh!

cursor bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

computermode Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

computermode Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

computermode Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

pfleidi commented Mar 31, 2026 •

edited by cursor bot

Loading

cursor bot left a comment •

edited

Loading